Regularization parameter estimation for feedforward neural networks

Studies of model selection and regularization for generalization in neural networks with applications

This thesis investigates the generalization problem in artificial neural networks, attacking it from two major approaches: regularization and model selection. On the regularization side, under the framework of Kullback–Leibler divergence for feedforward neural networks, we develop a new formula for the regularization parameter in Gaussian density kernel estimation based on available training da...

Tikhonov Regularization for Long Short-Term Memory Networks

It is a well-known fact that adding noise to the input data often improves network performance. While the dropout technique may be a cause of memory loss, when it is applied to recurrent connections, Tikhonov regularization, which can be regarded as the training with additive noise, avoids this issue naturally, though it implies regularizer derivation for different architectures. In case of fee...

Parameter-estimation-based learning for feedforward neural networks: convergence and robustness analysis

Application of Two Methods of Artificial Neural Network MLP, RBF for Estimation of Wind of Sediments (Case Study: Korsya of Darab Plain)

The lack of sediment gauging stations in the process of wind erosion, caused of estimate of sediment be process of necessary and important. Artificial neural networks can be used as an efficient and effective of tool to estimate and simulate sediments. In this paper two model multi-layer perceptron neural networks and radial neural network was used to estimate the amount of sediment in Korsya o...

Dynamics of Structural Learning with an Adaptive Forgetting Rate

Structural learning with forgetting is a prominent method of multilayer, feedforward neural network complexity regularization. The level of regularization is controlled by a parameter known as the forgetting rate. The goal of this paper is to establish a dynamical system framework f o r the study of structural learning both to ofSer new insights into this methodology and to potentially provide ...